Papers with recognizing textual entailment
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference (N18-1)
Copied to clipboard
| Challenge: | et al., 1996, show that many of the most actively studied problems in NLP depend in large part on natural language understanding (NLU). |
| Approach: | They propose a dataset for machine learning that uses ten different genres of English to evaluate sentences for their meanings. |
| Outcome: | The multi-genre natural language inference corpus is one of the largest available for natural language understanding. |
Language Models for Lexical Inference in Context (2021.eacl-main)
Copied to clipboard
| Challenge: | Lexical inference in context (LIiC) is a variant of the natural language inference task focused on lexical semantics. |
| Approach: | They propose three approaches based on pretrained language models for LIiC . they propose a few-shot NLI classifier and a relation induction approach based upon handcrafted patterns expressing the semantics of lexical inference. |
| Outcome: | The proposed approaches outperform the previous state of the art and show their potential for LIiC. |
Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment (L18-1)
Copied to clipboard
| Challenge: | a method to improve the quality of training data is needed . annotation errors of dialog act corpus mislead learning results of Bayesian network . |
| Approach: | They propose to introduce a null hypothesis for predictability of textual entailment labels and test it using a Naive Bayes model. |
| Outcome: | The proposed method does not reject the null hypothesis, but it improves on the existing models. |
Figurative Language in Recognizing Textual Entailment (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing RTE models struggle to capture figurative language, despite its ubiquity, it remains a bottleneck in automatic text understanding. |
| Approach: | They propose to frame five existing figurative language datasets into over 12,500 RTE examples. |
| Outcome: | The proposed models struggle to perform pragmatic inference and reasoning about world knowledge. |
On the Importance of Delexicalization for Fact Verification (D19-1)
Copied to clipboard
| Challenge: | Neural networks (NNs) perform state-of-the-art (SOA) performance in many complex tasks. |
| Approach: | They investigate the importance that a model assigns to various aspects of data . they experiment with two strategies of masking to mitigate this dependence on lexicalized information . |
| Outcome: | The proposed model improves on the in-domain dataset by 10% compared to the fully lexicalized model. |
FLUTE: Figurative Language Understanding through Textual Explanations (2022.emnlp-main)
Copied to clipboard
| Challenge: | Figurative language understanding is a recognizing textual entailment task, but lacks data for figurative language. |
| Approach: | They propose to use a dataset to analyze figurative NLI instances with explanations to improve models' performance. |
| Outcome: | The proposed dataset can scale up models even for figurative language using human annotations. |
SCENE: Self-Labeled Counterfactuals for Extrapolating to Negative Examples (2023.emnlp-main)
Copied to clipboard
| Challenge: | Manually collecting challenging negative examples can be costly and domain-specific. |
| Approach: | They propose a method that synthesizes negative examples from only positive ones using a mask infilling model. |
| Outcome: | The proposed method closes 69.6% of the performance gap on a dataset where half of the evaluation examples are unanswerable compared to a model trained on SQuAD 2.0. |
Beyond Recognising Entailment: Formalising Natural Language Inference from an Argumentative Perspective (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for recognizing textual entailment lack a standardized definition of inference, making it difficult to compare methods trained on different datasets. |
| Approach: | They propose a rigorous approach to align entailment recognition with argumentation theory by using a tool to assist humans in annotating arguments according to the PTA. |
| Outcome: | The proposed model is based on a human-trained dataset and provides insights into non-expert annotator training. |
Reversing Gradients in Adversarial Domain Adaptation for Question Deduplication and Textual Entailment Tasks (P19-1)
Copied to clipboard
| Challenge: | Existing domain adaptation techniques for question deduplication and RTE focus on transferring category independent knowledge between domains. |
| Approach: | They propose to use gradient reversal to explicitly learn shared and unshared (domain specific) representations between two textual domains to compensate for domain mismatch while distilling domain specific knowledge. |
| Outcome: | The proposed approach outperforms other methods on question deduplication and on recognizing textual entailment tasks, while still distilling domain specific knowledge. |